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USTING OF CLAIMS: 

Claim 1 (Currently Amended): An infonnation input processing computer system for mapping 
gestures to keys of an invisible virtual keyboard, the system comprising one or several cameras, 
one or more memories with CPU coimected to the cameras, and processes running in the CPU 
that associate gesture movements made without touching any touch sensors with typing and 
produce gesture associated textual output, wherein said processes capture gesture images, 
classify the gesture images into classes, associate each of the classes with one or more possible 
keys of the invisible keyboard, for each of said classes, assign a probability to each of the 
possible keys associated with the class, and integrate the probabilities assigned to the possible 
keys to identify a word for a sequence of gesturcsf.] : and 

wherein the processes that integrate the probabilities use one or more of the following: 

a) language module component that estimate probabilities of word strings 
corresponding to key candidate sequences: 

b) character frequency module that estimate probabilities of character strings 
corresponding key candidate sequences: 

ci confusable matrix that estimate how often correct gesture classes are confusable 

with another gesture classes: 
d) gesture classes probability model that estimate probability of observing a string of 

gesture classes given a sequence of gesture frames: 
el computation of a probability of production a sequence of keys given a string of 

gesture frames: 

0 generation of a lattice of sequences of keys given sequence of gesture frames; 
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g) finding the most probable sequ^ce of kevs from the lattice of key candidate 
strings. 

Claim 2 (Original): The gesture-key mapping system as in claim 1, where a feedback is 
provided to the user on what kind of keys are associated with the user's gestures. 

Claim 3 (Original): The gesture-key mapping system as in claim 2, where the feedback is 
provided using one or more of the following: displaying keys on a display, playing sounds labels 
for keys, displaying image indicators on a display, playing special sound indicators, projecting 
the keyboard to any surface,- and displaying picture of the keyboard with user's hands. 

Claim 4 (Currently Amended): An information input processing, gesture-key mapping computer 
system for generating text from hand gestures of a xiser relative to an invisible virtual keyboard, 
the system comprising one or several cameras, one or more memories with CPU connected to the 
cameras, and processes running in the CPU that associate gesture movements with typing and 
produce gesture associated textual output, where the gesture-key processing is provided using the 
following modules: 

a) a gesture capturing module that captures gestures, relative to an invisible 
keyboard and made without touching any touch sensors, through camera 
sensors; 

b) a gesture classificator module that classifies the gestures into; 
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c) an associator module for associating each of the classes with one or more 
possible keys of the invisible keyboard^ for eadi of said classes, assign a 
probability to each of the possible keys associated with the class; and 

d) an integrator module that integrates the probabilities assigned to the possible 
keys to identify a word for a sequence of gesturesr.] : and 

wherein the integrator module includes one or more of the following: 

a) language module component that estimate probabilities of word strings 

corresponding to key candidate sequences; 
b} character frequency module that estimate probabilities of character strings 

corresponding key candidate sequences: 
cX confusable matrix that estimate how often correct gesture classes are confusable 

with another gesture classes: 
dl gesture classes probability model that estimate probability of observing a string of 

gesture classes given a sequence of gesture frames: 
el computation of a probability of production a sequence of keys given a string of 

gesture frames; 

fl generation of a lattice of sequences of keys given sequence of gesture frames: 
gl finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 5 (Cancelled). 
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Claim 6 (Gurrently Amended): A system according to Claim & 4, wherein each sequence of keys 
receives a probability score, and the sequences of keys that receive low scores are removed and 
are not continuing when new candidates for keys arrive. 

Claim 7 (Previously Presented): A system according to Claim 4, further comprising a gesture 
correlator module to allow to adjust automatically the invisible keyboard to changes in the hand 
positions of the user, and wherein die gesture correlator module acts between the gesture 
classificator module and the associator module to maintain a consistent mapping of gesture 
classes to keys of the invisible keyboard despite said changes in the hand positions of the user. 

Claim 8 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gestur e gestures relative to an invisible virtual keyboard made without touching any 
touch sensors and without the presence of a real keyboard and the gestures are associated with 
the most probable keys that Would be typed if a keyboard were presented, said method including 
the steps of using a computer system to map gestures made, without touching any touch sensors, 
to keys of the virtual keyboard, including the steps of running processes on the computer to 
capture gesture images, to classify the gesture images into classes, to associate each of the 
classes with one or more possible keys of the invisible keyboard, for each of said classes, to 
assign a probability to each of the possible keys associated with the class, and to integrate the 
probabilities assigned to the possible keys to identify a word for a sequence of gesturesFJ : and 
wherein the processes to integrate the probabiUties use one or more of the following: 
a) language module component that estimate probabihties of word strings 
corresponding to key candidate sequences: 
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b} character frequency module that estimate probabilities of character strings 
corresponding key candidate sequences: 

c) confiisable matrix that estimate how often correct gesture classes are confasable 
with another gesture classes: 

d) gesture classes probability model that estimate probability of obs^rying a string of 
gesture classes given a sequence of gesture frames: 

ei computation of a probability of production a sequence of keys given a string of 
gesture frames: 

fx generation of a lattice of sequences of keys given sequence of gesture frames: 
gi finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 9 (Currently Amended): The method for producing a textual output in which a user makes 
typing like gestures relative to an invisible virtual keyboard and without the presence of a real 
keyboard and the gestures are associated with the most probable keys that would be typed if a 
keyboard were presented, said method including the step of using a computer system to map 
gestures made, without touching any touch sensors, to keys of the virtual keyboard, including the 
step of running processes on the computer to capture gesture images, to classify the gesture 
images into classes, to associate each of the classes with one or more possible keys of the 
invisible keyboard, for each of said classes, to assign a probability to each of the possible keys 
associated with the class, and to integrate the probabilities assigned to the possible keys to 
identify a word for a sequence of gestures, and wherein the probability is computed using 
HMMr.l: and 
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. wherein the processes to integrate the probabilities use one or more of the following: 

a} language module component that estimate probabilities of word strings 
corresponding to key candidate sequences: 

b) character frequency module that estimate probabilities of character strings 
corresponding key candidate sequences; 

c) confiisable matrix that estimate how often correct gesture classes are confiisable 
with another gesture classes: 

d) gesture classes probability model that estimate probability of observing a string of 
gesture classes giyen a sequence of gesture frames: 

e) computation of a probability of production a sequence of keys given a string of 
gesture frames: 

. U generation of a lattice of sequences of keys given sequence of gesture frames: 
gX finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 10 (Currently Amended): A method of typing using a virtual keyboard having a multitude 
of virtual keys^ comprising the steps: 

making typing gestures relative to an invisible virtual keyboard made without 
touching any touch sensors and without any real keyboard; 

sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output 
including the step of running processes on a computer to capture gesture images, to classify the 
gesture images into classes, to associate each of the classes with one or more possible keys of the 
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invisible keyboard, for each of said classes, to assign a probability to each of the possible keys 
associated with the class, and to integrate the probabilities assigned to the possible keys to 
identify a word for a sequence of gestures[.] : and 

wherein the processes to integrate the probabilities use one or more of the following: 
a) language module component that estimate probabilities of word strings 

corresponding to key candidate sequences: 
bl character frequency module that estimate probabilities of character strings 

corresponding key candidate sequences: 
c) confusable matrix that estimate how often correct gesture classes are confusable 

with another gesture classes: 
d} gesture classes probability model that estimate probability of observing a string of 

gesture classes given a sequence of gesture frames: 
ei computation of a probability of production a sequence of keys given a string of 
gesture frames: 

0 generation of a lattice of sequences of keys given sequence of gesture frames: 
£l finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 1 1 (Original): A method according to Claim 10, wherein the typing gestures are made by 
a person, and ftirther comprising the steps of providing feedback to the person on texture output 
associated with the gestures. 
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c. 

Claim 12 (Original): A method according to Claini 11, \vhereia the step of providing feedback 
includes the step of displaying an image of typing keys associated with the gestures. 

Claim 13 (Cancelled). 

Claim 14 (Currently Amended): A method of typing using an invisible virtual keyboard, 
comprising the steps: 

making typing gestures relative to an invisible virtual keyboard made without 
touching any touch sensors and without any real keyboard; 

sensing the typing gestures; and 

producing, from the sensed typing gestures, gesture associated textual output: and 
wherein the producing step includes the steps of classifying the gestures into classes, associating 
each of said classes with one or more possible keys of the invisible keyboard, for each of said 
classes, assign a probability to each of the possible keys associated with the class, and integrating 
the probabilities assigned to the possible keys to identifying a word for a sequence of gestures[.]; 
and 

wherein the step of integrating the probabilities includes the step of using one or more of 
the following: 

a) language module component that estimate probabilities of word strings 
corresponding fo key candidate sequences: 

b) character frequency module that estimate probabilities of character strings 
corresponding key candidate sequences; 
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cl confusable matrix that estimate how often correct gesture classes are confusable 
with another gesture classes: 

d) gesture classes probabiHtv model that estimate probability of observing a string of 
gesture classes given a sequence of gesture frames: 

e) computation of a probability of production a sequence of keys given a string of 
gesture frames: 

fx generation of a lattice of sequences of keys given sequence of gesture frames: 
gl finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 15 (Original): A method according to Claim 14, wherein the producing step further 
includes the step of associating gesture classes with individual typing keys. 

Claim 16 (Previously Presented); A method according to Claim 14, further comprising 
providing training data in words or sentences with certain timing data. 

Claim 17 (Currently Amended): A typing system using an invisible keyboard, comprising; 

means for sensing typing gestures made relative to an invisible virtual keyboard 
made without touching any touch sensors and without any real keyboard; and 

means for producing, from the sensed typing gestures, gesture associated textual 
output said producing means including a computer and processes running on the computer to 
capture gesture images, to classify the gesture images into classes, and to associate each of the 
classes with one or more possible keys of the invisible keyboard, for each of said classes, to 
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assign a probability to each of the possible keys associated with the class, and to integrate the 
probabilities assigned to the possible keys to identify a word for a sequence of gesturesf.] : and 
wherein the processes to integrate the probabilities use one or more of the following: 
a} language module component that estimate probabilities of word strings 

corresponding to key candidate sequences: 
b} character frequency module that estimate probabilities of character strings 

corresponding key candidate sequences: 
cl confiisable matrix that estimate how often correct gesture classes are confusable 

with another gesture classes: 
d) gesture classes probability model that estimate probability of observing a string of 

gesture classes given a sequence of gesture frames: 
ei computation of a probability of production a sequence of keys given a string of 
gesture frames: 

fi generation of a lattice of sequences of keys given sequence of gesture frames; 
g) finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 1 8 (Original): A system according to Claim 17, wherein the typing gestures are made by a 
person, and ftirther comprising means for providing feedback to the person on texture output 
associated with the gestures. 

Claim 19 (Original): A system according to Claim 18, wherein the means for providing 
feedback includes means for displaying an image of typing keys associated with the gestures. 
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Claim 20 (Cancelled). 

Claim 21 (Currently Amended): A typing system using an invisible virtual keyboard, 
comprising: 

means for sensing typing gestures made relative to a virtual, invisible keyboard 
made without touching any touch sensors and without any real keyboard; and 

means for producing, from the sensed typing gestures, gesture associated textual 

output; and 

wherein the producing means includes means for classifying the gestures iuto 
classes, for associating each of said classes with one or more possible keys of the invisible 
keyboard, for each of said classes, assign for assigning a probability to each of the possible keys 
associated with the class, and for integrating the probabilities assigned to the possible keys to 
identifying a word for a response of gesturesF.] : and 

wherein the means for inte j grating the probabilities includes one or more of the following: 
a) language module component that estimate probabilities of word strings 

corresponding to key candidate sequences: 
bl character frequencv modxile that estimate probabilities of character strings 

corresponding key candidate sequences: 
c) confusable matrix that estimate how often correct gesture classes are confusable 

with anodier gesture classes: 
dl gesture classes probability model that estimate probability of observing a string of 
gesture classes given a sequence of gesture frames: 
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e) computation of a prpbabiiitv of production a sequence of keys given a string of 
gesture frames; 

fl generation of a lattice of sequences of keys given sequence of gesture frames: 
gi finding the most probable sequence of keys from the lattice of key candidate 
strings. 

Claim 22 (Original): A system according to Claim 21, wherein the producing means further 
includes means for associating gesture classes with individual typing keys. 

Claim 23 (Previously Presented): A method according to Claim 10, wherein the step of running 
processes on the computer includes the step of mapping the sensed typing gestures to keys of the 
keyboard based on a statistical machine that interprets sequences of typing like gesture classes as 
intended words based on user typing gesture models. 

Claim 24 (Previously Presented): A method according to Claim 10, wherein gesture input is 
represented as wave forms that are digitized and clxistered in gesture frames, and then processed 
by HMM machines that represent difference models for different typing patterns; and the step of 
running processes includes the step of using two statistical components, a first statistical 
component to map gestures to keys, and a second, language model component to map keys to 
letters and words. 
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